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ABSTRACT 

The Computerized Observation System (COS) is a 
software program which an observer can use with a portable 
microcomputer to document preservice and inservice teacher 
performance. Specific observable behavior such as appropriate 
questions and responses shown to increase student achievement are 
recorded as Low Inference Observation Measures. Time on task fcr 
student and teacher are also Low Inference Measures. The High 
Inference Rating Scale component of the COS measures a broad spectrum 
of teacher behavior currently used in teacher evaluation including 
enthusiasm, clarity, pacing, organization, oral communication, 
assessment, lesson objectives, behavior management, positive learning 
climate, and instructional methods. The Narrative Component is 
provided to record any pertinent information about the observation or 
setting that may be necessary for data collection, documentation, or 
recordkeeping purposes. The IBM convertible microcomputer was used 
for a field test of the program during the winter term of 1987, which 
was conducted with nine inservice teachers enrolled at Valdosta 
(Georgia) State College School of Education in a practician on 
teaching students with behavior disorders. Discussions of the 
advantages and disadvantages of the IBM convertible and preliminary 
results of the field test conclude the report. Three references and 
lists of the goals of the school of education, the observer, and the 
teacher are included. (MES) 
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PRESENTRTION OUTLINE 



I. COMPUTERIZED OBSERURTION SYSTEM (COS) 
COMPONENTS 
R. PULLDOUJN MENUS 

1. CLRSS ROLL 

2. LOW-INFERENCE MEvHSURE 

a. STRTUS SCREEN 

b. TERCHER ON TRSK 

c. STUDENT ON TRSK 

3. HIGH-INFERENCE RRTING SCALE 

4. NRRRRTIUE COMPONENT 
B. PROGRRM FERTURES 

1. HELP SCREENS 

2. TIMING 

3. PRINTOUTS 

4. DRTR MRNRGEMENT 



IL FIELD USE 

R. HRRDWRRE DESCRIPTION 

B. RRM DISK 

C. CURRENT STRTUS 

D. STRTISTICRL RNRLYSIS 



III. COS GORLS 

R. SCHOOL OF EDUCRTION GORLS 

1. NCRTE FIELD EKPERIENCE STRNDRRO 

2. SCHOOL OF EDUCRTION NETWORKING 

B. OBSERUER GORLS 

C. TERCHER GORLS 



ICDS) — r.awnf»t% nmwk*9t 

The Computerized Obseruation System (COS) is a software 
program accompanied by documentation that addresses the 
• ualuation of uarious aspects of field enperiences and student 
teaching. The COS is primarily designed for use u;ith preseruice 
teachers in field enperiences or preseruice teachers in student 
teaching. It may also be used to obserue enperienced teachers 
inuolued in practicum eHperiences at the graduate ieuel. 

PMllaljiMtm M«»mma 



Each component of the COS is accessed through puiidou^n 
menus. After the observer performs an observational task such 
as counting discrete behaviors or luriting suggestions to the 
teacher, he or she is returned to the pulldou^n menu for further 
action. This type of control ouer the COS program allou^s the 
observer an easy method of moving from one component to 
another. 

Class Roll 



Each time the COS program is started, the observer is 
requested to select the teacher to be observed. Since most 
observations of teachers in a higher education setting are 
receiving course credit for their practice teaching, the COS 
program provides a module u^hich allou^s the recording of ail 
teachers, preservice or inservice, enrolled in a specific course, 
along u^ith the type of practicum setting, the grade level of the 
students, the school u^here the practicum is occurring, and the 
city u^here that school is located. Once this information is typed 
into the computer, it is saved on diskette so thait the observer 
only needs to type the information once. After the teacher to be 
observed is selected from the class list, the observer can select 
from the three basic components of the COS; the Lou^-lnf er ence 
Observation Measure, the High-Inference Rating Scale, or the 
Narrative Component. 
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Loiif-Inf erence Obsc»ruation Measure 



The teaching behauiors identified in the Lou;~lnf erence 
Obserti^ation Measure are meant to prouide teachers u;ith 
conc?0rii4rated practice in prouiding students u;ith quality 
instruction. fl description of the lou;-inf erence teacher bi^hauiors 
is presented using current research findings and practicrjl 
eHamples. Specific obseruable behauiors that haue been shou^n to 
increase student achieuement are singled out, so as to giue 
teachers the opportunity to master these important behauiors. 
The 1ou;-inf erence teacher behauiors include: 

- Rssociates neiiu learningi u/^ith PREREQUISITE learning. 
~ States appropriate GORLS of instruction. 

- Prouides appropriate SMRLL STEPS of instruction. 

- Rsks clear apprcipriate QUESTIONS. 

- Prouides appropriate FEEDBRCK to CORRECT student responses. 

- Prouides appropriate FEEDBACK to INCORRECT student responses. 

- Prouides for OUERLERRNING. 



Student and Teachc»r On Task Behau«or 

Tluo other lou;-inf erence behauiors measured by the COS 
are teacher on task and student on task . Their measurement 
differs from the preuious behauiors in that the obseruer has the 
option to choose measurement of teacher on task, student on 
task, or' both teacher on task and student on task. The obseruer 
also chooses the time interual for monitoring one or both of the 
behauiors. For ewample if the obseruer chose teacher on task 
and 30 seconds, the computer u^ould ask "Is the teacher on task?' 
euery 30 seconds and delay the obseruation program until the 
question u^as answered. The resulting obseruation measure uiould 
compare the number of times the teacher u;as on task, in the 
judgement of the obseruer, u^ith the number of times the question 
mas asked during the obseruation period. 
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High-Inference Rating Scale 



The High-Inference Rating Scale component of the COS 
measures a broad spectrum of teaching behauiors currently used 
to eualuate teachers. They are included to address areas of 
teacher eualuation not couered by the Loui^inf er ence Obseruataon 
Measure and to alloui each obseruer using the COS to indii'sdualize 
the program. The obseruer accomplishes this by focusing on the 
high-inference behauiors that relate directly to practicum 
objectiues and perceiued teacher needs uihile deleting unnecessary 
behauiors from the scale. The obseruer can then communicate 
the chosen behauiors to the teacher prior to the first 
obseruation. This ailoLus the teacher to concentrate on those 
behauiors germane to the'^r program and indiuidual needs. 
The teacher being obserued has the opportunity to enpress their 
opinion as to ujhether or not they uiere capable of their best 
performance ujithin the limited obseruation period by rating 
themselues on the same scale used by the obseruer. The 
high-inference teacher behauior'^ currently being eualuated include: 

Enthusiasm 

Clarity 

Pacing 

Organization 

Oral Communication 

Assessment 

Lesson Objectiues 

Behauior Management 

Pbsitiue Learning Climate 

Instructional Methods 

Narrative Component 

The Narratiue Component of the COS is prouided to record 
any pertinent information about the obseruation or setting that 
mi'iy be necessary for data collection, documentation, or record 
keeping purposes. Possibilities include social economic ^status of 
the school, comments on the feedback giucn the teacher, or any 
unusual circumstances of the obseruation such as a fire drill in 
the middle of an obser ua tional period. The Narratiue Component 
may also be used to elaborate on obseruatioba results, reteach 
course objectiues or to prouide suggestions for improuement to 
the teacher. 
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The Computerized Obseruation System (COS) has a number 
of features u;hich assist the obseruer in accurately recording 
the instructional lessons. these features include the eHtensiue 
use of help screens to clarify portions of the program and 
efficient use of the microcomputer's timing mechanisms. 

Help Screens 

Throughout the program, the obseruer can request 
clarification of specific program commands by typing the Function 
key "F1.*' The screen clears and information conc^^rning the 
questions the obseruer may haue is presented. After the 
obseruer reads the information, he or she is returned to the 
program. The use of the help screens allou^s obseruers to begin 
using the program u^ith minimum difficulty. It also eliminates the 
reliance on printed documentation. 

Timing 

The MS-DOS portable microcomputers use an internal clock 
to record the date and time. The Computerized Obseruation 
System uses this feature in the low-inference component to 
record actual obseruation time. Eack^i time the obseruer requests 
assistance from the help screen or desires to uieu; the current 
status of the obseruation, time is suspended until the obseruer 
returns to record lou^-inf er ence teacher behauiors. The obseruer 
can also leaue the lou^-inf er ence component and use either the 
high-inference or narratiue component u^ith the obserued time in 
the lou^-inf er ence section continuing only u^hen the obseruer 
returns to that section. By counting the observed time only u^hen 
the obseruer actually has the opportunity to record a 
lou^-inf er ence teaching behauior, the results of those obseruations 
u^tth obseruers less familiar u^ith the COS can be compared u^ith 
those using the system more frequently. Additionally, 
comparisons are enhanced because uarious frequencies of 
teacher behauiors can be transformed to incidents per minute so 
that the length of obseruation periods can uary and the 
frequency of behauiors still be compared. 
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Data management 

The results of each obseruation are saued into a 
lou^'inf erence and high inference data file. UJith the 
lom-inf erence file, demographic information stored includes the 
teacher's student number, the date of the obseruation, the 
subject of the lesson taught, the size of the class, the number 
of minority and speccal education students, and the size of the 
instrur tional group. In addition, the frequency of each teaching 
behauior, the frequency of correct and incorrect student 
responses at three minute internals, the total time that the 
obseruer u;as recording behauiors during the lesson, and student 
and teacher time on task are stored for each lesson obserued. 
To date, appr onimately 20 obseruations are being added to the 
loii;-inf erence data file each meek. 

The high inference data file contains the teacher's student 
number, the date, and the rating of each high inference behauior 
rated by the obseruer and the teacher. The obseruer may elect 
to complete the rating scale any time during or after an 
obseruation. 

FIEI n USF 

Hardware 

The IBM conuertible microcomputer u;as selected for field 
testing the Computerized Obseruation System <COS). Other MS-DOS 
portables u^-M also run the COS softujare program, including 
Zenith's models 171 and 181 as u.*ell as Toshiba's TllOO and TllOO 
Plus. 

Rdffitions to the conuertible's basic unit included memory 
eMpansion from 256K to 512K, a parallel/serial adaptor, and a 
printer. The parallel/serial adaptor prouides acjess to most 
printers and to telecommunications deuices. The class list, 
results of the lou^-inf erence obseruation, high~inf erence rating 
scale eualuations, and narratiue comments can be printed by 
connecting the conuertible to a printer or using the printer 
specifically designed for the conuertible. The data files haue 
been stored as tent (or ASCII) files so that the obseruer can use 
other database or statistical softu^are packages to analyze the 
results of the obseruations. In addition, the program diskette 
contains seueral files u^hich contain the control statements for 
the Statistical Package fcr the Social Sciences - uersion H 
(SPSSK). By uploading the obseruation data and the control file to 
a mainframe computer using SPSSH, uarious data analyses can be 
performed to inuestigate the relationships among obserued 
uariables. 
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RRM disk 

The memory enpansion alloujs for the creation of a RRM 
disk. R RRM disk emulates a disk driue u;ith disk access then 
directed io the RRM disk. The COS program can be copied into the 
RRM disk's memory along u;ith pertinent data files using a bat^h 
file. By setting up this copy routine prior to the obseruation 
period, all disk access during the obserued lessons ujili be to the 
RRM disk, thus eliminating any disk driue noise. Rt the completion 
of the obseruation period, u;hen disk driue noise u^ould not be 
distracting, the obseruer can transfer the collected data to the 
data diskette. 

Bdvantag^s and Disadvantages of the 
IBM Convertible 

One disaduantage of the neu; MS-DOS portables, eHcept for 
Zenith's model 171, is the use of 3 1/2 inch diskettes instead of 
the common 5 1/4 inch diskette. Users of the uarious MS-DOS 
portables haue indicated the difficulty of transferring files from 
the portable so that they could use the information on the 
uarious IBM and IBM compatible desktop models. UJhile some 
users haue installed a 3 1/2 inch disk driue into their desktop 
model, another possibility is using a softu^are package, such as 
Brooklyn Bridge. This package alloups rapid transfer 6f files 
through the serial ports of tu^o microcomputers. This softu^are 
package can help eliminate the disaduantage of different size 
storage media at minimal cost. 

Another disaduantage of the IBM conuertible microcomputer 
is the display screen. IBM's LCD screen has been rated lou; in 
readability u;hen compared to Zenith s or Toshiba's backlighted LCD 
screens (Unger, 1987). iDhile poorly rated, the difficulty of uieu^ing 
the conuertible's LCD screen can be considered a benefit in the 
classroom Luhen obseruing teachers, because only the obseruer is 
able to uieu> the screen clearly. 

Current status 

During the IDinter term, 1987, the Computerized Obseruation 
System (COS) has &een field tested u;ith nine inseruice teachers 
enrolled in a practicum for teaching students u^ith behauior 
disorders as ujell as a;ith one student teacher in a classroom 
setting seruing the mildly handicapped. Rll practicum teachers 
u^ere prouided u^ith a draft of the teacher documentation at the 
beginning of the term and indiuidual meetings u;ere arranged so 
that the teacher behauiors to be obserued u^ere clarified. 
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Statistical analyses 



During the first stage of the field testing, tu;o objectiues 
haue been identified that require statistical analysis and focus on 
the technical aspects of the obseruation system. These 
objectiues include: 

<1) identifying the most efficient method of presenting the results 
of the lou;-inf er ence component after an observation period, 

One method of displaying a teacher's performance is a 
profile analysis as described by Gibney and UJiersma (1986). The 
frequency of each teaching behauior can be compared u^ith other 
performances of that teacher or compared u^ith similar teachers 
at the same or different leuels of training using an area 
transformation as recommended by Medley, Coker and Soar (1984). 
These authors suggest that en comparing observational data that 
frequencies be converted to percentiles and then to standard 
scores. Rdditionally, in accurately conveying the teacher's 
performance, they caution the use of a normal distribution u^ith 
some frequency scores u^hich may not occur in each observation 
period. IJJith these suggestions affecting the data analysis, the 
presentation of results in the form of Ur charts, line graphs, 
or numeric tables u^ill be evaluated as to each form's clarity in 
communicating the teacher's performance. 

(2) determining the consistency of the measurements over the 
term. 

Medley, Coker, and Soar (1984) offer valuable suggestions 
in evaluating it<»ms in the performance measure. In developing 
and refining scoring keys ujhich provide teachers uiith the results 
of their performance, the authors indicate that a factor analysis 
of the lou^-inf erence items u^ill assist in identifying hou; to best 
combine items to convey results. fls the number of observations 
increase, such analyses u^ill be possible and scoring keys u^hich 
provide the most accurate and useful observational data u*ill be 
evaluated. 
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The primary focus of the obseruations has been the 
counting of the behauiors outlined in the lom-inf er ence component 
of the COS. Rll obseruations to date haue been collected by one 
indi: 'dual, the faculty member responsible for the superuision of 
the practicum eHperiences. 

Tu;o objectiues of the UJinter term field te<stmg include: 

(1) hauing the program run smoothly, 

H major objectiue of the COS is for the program to allou; 
any interested faculty member or other obseruer to use the 
program u;ith minimal tim'^ inuested in learning the system and 
yet hauing the data collected be reliable. To accomplish this 
objectiue, all menu selections need to be understandable and if 
that isn*t possible, the obseruer can access help A^creens that 
clarify any selection. Also, the obseruer should haue the 
fleKibility to moue betu^een program components easily. For 
eHample, if the obseruer is recording the frequency of teacher 
behauiors in the lou^-inf er ence component and u^ants to u^rite 
seueral suggestions to the teacher, the program allou^s the 
obseruer to leaue the lou^-inf er ence section, select the narratiue 
component, u^rite the suggestions, and return to the lou;-inf er ence 
measure and continue recording the occurrence of behauiors. For 
a second eHample, \f another student enters the instructional 
group, the obseruer can change the instructional group size and 
again continue the obseruation session. 

(2) measure student's distractions to the computer-based 
obseruation system. 

fl concern of both the teacher and the obseruer in using a 
microcomputer to record teacher performance is its distracting 
effect on students. To see the teacher performing optimally, it 
u^ould be unfair if students u^ere continually focusing their 
attention on the obseruer and microcomputer. Although 
acceptance of the portable computer in the classroom has been 
positiue, the recorded impressions of the obseruer and teachers 
being obserued u^ill prouide documentation as to the effects of 
the intrusion of the portable microcomputer in the classroom 
enuironment. 
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Schoal of Education fioaBs 

1- standard comparison measure for program eualuation and 
deuelopment 

2- documentaiion for NCHTE field enperience standard 

3- eualuation tool for course objectiues 

4- research data base auailable to faculty through netu^orking of 
departments 

Obs»rv»r Goals 

1- record obseruation data efficiently, document feedback, and 
organize record keeping 

2- create a data base of teacher performance for comparison 

3- improue feedback to teachers in the field by making it 
immediate, more complete, less threatening, and concrete 

4- increase student opportunity to learn by emphasizing 
increasing time spent in academic actiuities 

5- document teacher grou^th u^ithin individual practicum 
ewperiences and throughout professional programs 

6- prouide a standard of performance for teacher skill 
acquisition 

7- emphasis sequence of ^uents in instruction 
Teacher Goals 

1- prouides preparation for professional teacher eualuation 
measures 

2- prouides specific goals and objectiues for instruction 

3- emphasises mastery of basic teaching skills 

4- indiuidualizes according to perceiued needs 

5- allouis for teacher input and self eualuation 

6- allou^s for comparisons u^ith different standards of 
performance, for eMampie, preseruice teachers, student teachers, 
and post graduate teachers 
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